Careful consideration is paid to optimal performance when developing IP.Board, so modifying indexes alone is not likely to improve performance (else it would have been done already). Although I specialise in MSSQL, not MySQL, my understanding is that many semantics are the same.

There are nonetheless several things you can do to improve performance. The first is that you can install cache engines like Memcache, APC, or Xaccelerator. This means there are less hits to the database period, which can lend a significant boost for frequently used tables. Second if money is of little to no object is you can split reads and writes to different database servers or clusters. Although there's no built in functionality to do that, there is an article in the resources area describing how you can do it.

In response to your question of how many accesses per second can result in a table lock, the answer depends on your engine. My understanding is that MyISAM does not support row locking, which means when a lock occurs it must be a full table lock. These are obviously very bad. InnoDB apparently supports row level locking, which should reduce issues with concurrency, but it has the footprint of a brontosaurus for large tables - make sure you have lots of RAM!

Simple ways you can improve performance if you haven't already include using an engine like Sphinx for search on top of your DB, so search does not need to FULLTEXT scan the posts and topics tables. As a bonus, it has really good results quality. You can also enable "delayed view counter" so that IPB only updates the view count on a thread once an hour or so - as this activity requires a write to the topics table, it necessitates a lock (remember, that's a table lock if you use MyISAM) on every page view!